FeDaL: Federated Dataset Learning for General Time Series Foundation Models
Table of Contents
FeDaL focuses on training time series foundation models (TSFMs) under federated settings. In short, FeDaL decomposes heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge; it mitigates local and global biases through bias elimination. FeDaL also shows how data volume, client count and join rate affect model performance under decentralization.
1. Challenge
The challenge that FeDaL is trying to handle is the conflict between general-purpose foundation model for time series and heterogeneous and private datasets.
The heterogeneity consists of:
- Temporal resolution bias
- Physical constraint bias
- Pattern transition bias
2. Problem Formulation
The problem can be formulated as follow: consider a federated foundation model settings with \(1\) server and \(N\) clients, where each client \(i\) holds a local time series dataset \(D_i\). These \(D_{i}\) are heterogeneous due to difference in sampling resolution, physical constraints and temporal dynamics.
The objective is to train a general TSFM that generalizes. Formally, the global objective is
\[ F(\theta)=\mathop{\text{argmin}}\limits_{\theta} \sum_{i=1}^{N} \frac{n_{i}}{n} F_{i}(\theta_{i}; D_{i}) \]
where \(F_{i}\) denotes the local loss, \(n_i,n\) denotes local and total sample counts respectively.
3. TODO Solution
The core of FEDAL consists of client-side Domain Bias Elimination and server-side Global Bias Elimination.
3.1. Domain Bias Elimination (DBE)
DBE decomposes latent representations of masked input patches into trend and seasonal components, from which a trainable bias vector is learned. This vector is injected back during reconstruction, to suppress dataset-specific artifacts. Moreover, a regularization term aligns it with a global bias reference to disentangle non-transferable local patterns from generalizable temporal structures.